The so-called AB test is to develop two solutions (for example, two pages) for the same objective, so that a part of users uses the solution A, the other part of users uses the solution B, the use conditions of the users are recorded to determine which solution better satisfies the design objective.
In a live video website, in order to improve the user experience during live video watching, the website frequently adjusts products and systems through various methods and strategies, at this time, user feedback needs to be quickly obtained so that the effects of the new solution can be adjusted in time, and the solution is iteratively updated according to the feedback data of the users, and if the feedback results of the users on the solutions of the products and the systems are unsatisfactory, the current new solution even needs to be abandoned, so the AB test is very important in a live video system and is the core foundation of data-driven products.
In a general AB test process, two solutions are operated at the same time in parallel: Control (reference solution) and Treatment (experimental solution), the shorter the effective process of the experimental solution is, the more accurate the test of feedback of a group of users on the trial of the new solution. In the effective process of the experimental solution, the inflow and outflow of users in the website are completely unpredictable, and thus the effective time of the experimental solution needs to be shortened in the test. Moreover, for the live video website, the new solution is adjusted frequently and the AB test frequency is high, so the friendliness of using an AB test system is quite important, such that more non-technical persons may participate in the AB test process, and the error probability of the AB test strategy may be reduced.
In the live video website, the HTTP request of the user firstly enters a website gateway server, and then the gateway server allocates the HTTP request to a specific background service server for processing according to the IP range (users in a specific region) characteristics of the request. How to configure the IP strategy in the gateway server is especially important. In the existing AB test method, if an IP distribution strategy needs to be modified, a company business team specifies the geographical range of the users to limit that only the users in some regions can use new functions, and after the request is submitted to a technical team, the technical team outputs a configuration file content to an operation and maintenance team for operation, and the gateway server needs to be restarted, so that the gateway server becomes effective to the newly entering HTTP request, and if the newly configured IP range is hit, the HTTP request is distributed to the background server corresponding to the new solution for processing. Too many people are involved in the whole process, so that the distribution strategy is prone to errors, and the effective process is long.